Health technology assessment of online eLearning for post-registration health professionals’ education

Aim: The overall aim of this thesis was to undertake and report the findings of a health technology assessment (HTA) on the effectiveness, cost-effectiveness and acceptability of online and LAN-based eLearning, and blended learning, among post-registration healthcare professionals. Methods: This HTA comprised three studies. The first study was a systematic review of 93 randomised controlled trials evaluating the effectiveness of online and LAN-based eLearning on physicians' knowledge, skills, attitude and satisfaction. The second study compared the cost, cost-savings and return on investment between a blended and a face-to-face advanced cardiac life support course for physicians in Singapore. The third study was an online survey that assessed the acceptability of the technology among a sample of optometrists and opticians in Singapore and their scope of practice, primary eye care knowledge, views on extended roles in primary eye care, preferred mode of learning for continuing professional education, and referral behaviour. Results: The systematic review showed that online and LAN-based eLearning or blended learning compared with self-directed or face-to-face learning resulted in higher post-intervention knowledge scores (21 studies; small to large effect size; very poor quality); higher post-intervention skills scores (seven studies; large effect size; low quality); higher attitude scores (one study; very low quality); higher post-intervention satisfaction (four studies; large effect size; low quality); and higher post-intervention practice or behaviour changes (eight studies; large effect size; low quality) among physicians in the intervention groups. Fourteen studies compared eLearning with other forms of eLearning. Among these, four studies reported higher post-intervention knowledge scores (large effect size; very low quality) for participants in the intervention group. Unintended or adverse effects of the intervention were not reported among the included studies. Ninety-three studies (N=16,895) were included of which seventy-six studies compared ODE (including blended) vs self-directed/face-to-face learning. Overall the effect of ODE (including blended) on post-intervention knowledge, skills, attitude, satisfaction, practice or behaviour change and patient outcomes was inconsistent and ranged mostly from no difference between the groups to higher post-intervention score in the intervention group (small to large effect size, very low to low quality evidence). Twenty-one studies reported higher knowledge score (small to large effect size; very low quality) for the intervention while 20 studies reported no difference in knowledge between the groups. Seven studies reported higher skill score in the intervention (large effect size; low quality) while thirteen studies reported no difference in skill score between the groups. One study reported higher attitude score for the intervention (very low quality), while 4 studies reported no difference in attitude score between the groups. Four studies reported higher post-intervention physician satisfaction with the intervention (large effect size; low quality), while six studies reported no difference in satisfaction between the groups. Eight studies reported higher post-intervention practice or behaviour change for the ODE group (small to moderate effect size; low quality) while five studies reported no difference in practice or behaviour change between the groups. One study reported higher improvement in patient outcome, while three others reported no difference in patient outcome between the groups. None of the included studies reported any unintended/adverse effects, cost-effectiveness of the interventions. Although the review only focused on post-registration medical doctors, the technology could be used for the interprofessional education of post-registration medical doctors and other healthcare professionals. Such an initiative would encourage collaborative learning and facilitate task-shifting, which could address the problem of fragmentation in health care. Although eLearning and blended learning technology interventions have been implemented, primary studies have not assessed their cost-effectiveness. Hence, to ascertain the technology’s cost-saving potential, we used a blended advanced cardiac life support (B-ACLS) course as an exemplar and compared its cost to face-to-face advanced life support (F-ACLS) training. The analysis showed that the annual cost of F-ACLS training (USD$72,793) was 1.7 times higher than B-ACLS training (USD$43,467). The discounted total cost of training over the life of the course (5-years) was SGD $107,960 for B-ACLS and S$280,162 for F-ACLS. The cost of productivity loss accounted for 52% and 23% of the costs for F-ACLS and B-ACLS, respectively. B-ACLS yielded a 160% return on the money invested, yielding $1.60 for every dollar spent. There would be a 61% saving for course providers if they delivered a B-ACLS instead of F-ACLS course. The effectiveness component of the HTA showed that online eLearning and blended learning is as effective as traditional learning and has cost-saving potential. We also sought to determine if this technology could be used to train and equip optometrists and opticians in Singapore to take on an extended role in primary care, which would allow some simple primary eye care tasks to be shifted from ophthalmologists to optometrists and ease healthcare access issues at specialist hospital outpatient clinics. The survey of optometrists showed that the current roles of opticians and optometrists in Singapore were limited to diagnostic refraction (92%); colour vision assessment (65%); contact lens fitting and dispensing (62%) amongst others. The average self-rated primary eye care knowledge score was 8.2 ± 1.4; (score range 1-10; 1 = very poor, 10 = excellent). Average self-rated confidence scores for screening for cataract, diabetic retinopathy, chronic glaucoma and age-related macular degeneration were 2.7 ± 1.5; 3.7 ± 1.9; 4.0 ± 1.0 and 2.7 ± 1.5, respectively. Three fourths of the optometrists surveyed felt that they should undertake regular continuing professional education (CPE) to improve their primary eye care knowledge. Blended learning (eLearning and traditional face-to-face lectures) was the most preferred mode (46.8%) for CPE delivery. Conclusions: Overall, the findings from the HTA provide evidence of effectiveness, cost-saving of online eLearning and blended learning for training medical doctors and the acceptance of the technology in a local context to facilitate its wider adoption for training post-registration healthcare professionals’. These research outputs would have direct impact on the adoption of online eLearning, blended learning technologies in universities and educational institutes across the region with consequent impacts on post-registration health professionals’ education and policy. The results of learning will serve as a guide for policy makers to decide on investment in the learning technology and to learn about the associated factors, which would influence its adoption. This thesis resulted in three papers, of which one has been accepted for publication, the two other papers are under review.Thesis (Ph.D.) -- University of Adelaide, Adelaide Medical School, 201